Accelerating flat reinforcement learning on a robot by using subgoals in a hierarchical framework
نویسندگان
چکیده
Learning a motor skill task with Reinforcement Learning still takes a long time. A way to speed up the learning process without using much prior knowledge is to use subgoals. In this study, the use of subgoals decreased the learning time by a factor nine and we show that tests on a real robot give similar results. The price to be paid, in case the subgoals do not lie on the optimal path, is a worse end performance. Hierarchical greedy execution can (partially) cancel out this problem. For future work, we suggest the use of a method which is able to obtain optimal performance.
منابع مشابه
Learning State and Action Hierarchies for Reinforcement Learning Using Autonomous Subgoal Discovery and Action-Dependent State Space Partitioning
This paper presents a new method for the autonomous construction of hierarchical action and state representations in reinforcement learning, aimed at accelerating learning and extending the scope of such systems. In this approach, the agent uses information acquired while learning one task to discover subgoals for similar tasks. The agent is able to transfer knowledge to subsequent tasks and to...
متن کاملHierarchical reinforcement learning with subpolicies specializing for learned subgoals
This paper describes a method for hierarchical reinforcement learning in which high-level policies automatically discover subgoals, and low-level policies learn to specialize for different subgoals. Subgoals are represented as desired abstract observations which cluster raw input data. High-level value functions cover the state space at a coarse level; low-level value functions cover only parts...
متن کاملAccelerating Reinforcement Learning through the Discovery of Useful Subgoals
An ability to adjust to changing environments and unforeseen circumstances is likely to be an important component of a successful autonomous space robot. This paper shows how to augment reinforcement learning algorithms with a method for automatically discovering certain types of subgoals online. By creating useful new subgoals while learning, the agent is able to accelerate learning on a curre...
متن کاملHierarchical Reinforcement Learning Based Self-balancing Algorithm for Two-wheeled Robots
Abstract: Self-balancing control is the basis for applications of two-wheeled robots. In order to improve the self-balancing of twowheeled robots, we propose a hierarchical reinforcement learning algorithm for controlling the balance of two-wheeled robots. After describing the subgoals of hierarchical reinforcement learning, we extract features for subgoals, define a feature value vector and it...
متن کاملSubgoal Discovery for Hierarchical Reinforcement Learning Using Learned Policies
Reinforcement learning addresses the problem of learning to select actions in order to maximize an agent’s performance in unknown environments. To scale reinforcement learning to complex real-world tasks, agent must be able to discover hierarchical structures within their learning and control systems. This paper presents a method by which a reinforcement learning agent can discover subgoals wit...
متن کامل